<html>
<head><meta charset="utf-8"><title>terraform with rds in a private subnet · t-infra · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/index.html">t-infra</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html">terraform with rds in a private subnet</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="207523918"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207523918" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207523918">(Aug 20 2020 at 13:48)</a>:</h4>
<p>so, I've been trying to think how to solve connecting within terraform to postgresql, when the instance is managed by rds in a private subnet</p>



<a name="207524019"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524019" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524019">(Aug 20 2020 at 13:49)</a>:</h4>
<p>to connect to it from the outside we need to pass through the bastion host, ideally with ssh</p>



<a name="207524093"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524093" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524093">(Aug 20 2020 at 13:50)</a>:</h4>
<p>there are two approaches that come to mind</p>



<a name="207524197"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524197" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524197">(Aug 20 2020 at 13:51)</a>:</h4>
<p>one is the simplest: we have a <code>./proxy.sh</code> script that setup ssh forwarding with an hardcoded list of database instances -- with this approach, before running terraform we'd have to start the script in another terminal, run terraform and then kill the script</p>



<a name="207524213"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524213" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524213">(Aug 20 2020 at 13:51)</a>:</h4>
<p>every time the connection urls change we'd also need to change the script, which is far from ideal</p>



<a name="207524351"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524351" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524351">(Aug 20 2020 at 13:53)</a>:</h4>
<p>the second approach is way more hacky, but it requires way less maintenance and it "just works"</p>



<a name="207524496"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524496" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524496">(Aug 20 2020 at 13:54)</a>:</h4>
<p>using <a href="https://registry.terraform.io/providers/hashicorp/external/latest/docs/data-sources/data_source">the external data source</a> we start a script that:</p>
<ol>
<li>receives the connection params from terraform</li>
<li>starts the ssh port forwarding on a random port and daemonizes the ssh process</li>
<li>returns the used port to terraform</li>
</ol>



<a name="207524670"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524670" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524670">(Aug 20 2020 at 13:56)</a>:</h4>
<p>then after a timeout (configurable) the connection will be killed</p>



<a name="207524684"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524684" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524684">(Aug 20 2020 at 13:56)</a>:</h4>
<p><span class="user-mention" data-user-id="121055">@Pietro Albini</span> could we have terraform configure rds to allow public ip for the current ip, run terraform, and then revert that?</p>



<a name="207524699"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524699" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524699">(Aug 20 2020 at 13:56)</a>:</h4>
<p>but whatever works seems fine</p>



<a name="207524707"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524707" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524707">(Aug 20 2020 at 13:56)</a>:</h4>
<p><span class="user-mention" data-user-id="116122">@simulacrum</span> if it's on the private subnet it's not possible to reach it from the outside at all</p>



<a name="207524723"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524723" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524723">(Aug 20 2020 at 13:57)</a>:</h4>
<p>ah right</p>



<a name="207524730"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524730" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524730">(Aug 20 2020 at 13:57)</a>:</h4>
<p>the downsides of this approach are that the connection lingers until the timeout is reached</p>



<a name="207524761"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524761" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524761">(Aug 20 2020 at 13:57)</a>:</h4>
<p>but it means <code>terraform plan/apply</code> just works</p>



<a name="207524822"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524822" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524822">(Aug 20 2020 at 13:58)</a>:</h4>
<p>I think just working is pretty important, otoh we already need to do the aws auth</p>



<a name="207524825"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524825" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524825">(Aug 20 2020 at 13:58)</a>:</h4>
<p>so maybe not <em>that</em> important</p>



<a name="207524854"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524854" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524854">(Aug 20 2020 at 13:58)</a>:</h4>
<p>the big downside of that approach is that we have to manually keep the hostname to conect to up to date in the separate script</p>



<a name="207524868"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524868" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524868">(Aug 20 2020 at 13:58)</a>:</h4>
<p>and if we add more instances it's going to become a mess</p>



<a name="207524937"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524937" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524937">(Aug 20 2020 at 13:59)</a>:</h4>
<p>eh we can have terraform update a dns entry or whatever, seems fine</p>



<a name="207524946"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207524946" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207524946">(Aug 20 2020 at 13:59)</a>:</h4>
<p>I'm wondering if <span class="user-mention" data-user-id="225192">@Nell Shamrell-Harrington</span> (or others using terraform) encountered something similar</p>



<a name="207525009"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207525009" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207525009">(Aug 20 2020 at 14:00)</a>:</h4>
<p>online it seems people either setup vpns or do something similar to the second option</p>



<a name="207554356"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207554356" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207554356">(Aug 20 2020 at 18:02)</a>:</h4>
<p>ok managed to have it do the forwarding transparently in the background <span aria-label="tada" class="emoji emoji-1f389" role="img" title="tada">:tada:</span></p>



<a name="207556091"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/207556091" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#207556091">(Aug 20 2020 at 18:18)</a>:</h4>
<p><a href="https://github.com/rust-lang/simpleinfra/pull/49">https://github.com/rust-lang/simpleinfra/pull/49</a> <span aria-label="tada" class="emoji emoji-1f389" role="img" title="tada">:tada:</span> <span aria-label="tada" class="emoji emoji-1f389" role="img" title="tada">:tada:</span> <span aria-label="tada" class="emoji emoji-1f389" role="img" title="tada">:tada:</span></p>



<a name="208810870"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208810870" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208810870">(Sep 02 2020 at 11:40)</a>:</h4>
<p><span class="user-mention" data-user-id="116122">@simulacrum</span> fyi, trying to put rds in the private subnet</p>



<a name="208810890"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208810890" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208810890">(Sep 02 2020 at 11:40)</a>:</h4>
<p>Ok!</p>



<a name="208810919"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208810919" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208810919">(Sep 02 2020 at 11:40)</a>:</h4>
<p>I forget if I commented on the PR but I did test the script and it worked fine for me</p>



<a name="208810945"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208810945" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208810945">(Sep 02 2020 at 11:41)</a>:</h4>
<p><span aria-label="thumbs up" class="emoji emoji-1f44d" role="img" title="thumbs up">:thumbs_up:</span></p>



<a name="208811041"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208811041" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208811041">(Sep 02 2020 at 11:42)</a>:</h4>
<p>no wait</p>



<a name="208811043"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208811043" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208811043">(Sep 02 2020 at 11:42)</a>:</h4>
<p>#!@</p>



<a name="208811079"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208811079" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208811079">(Sep 02 2020 at 11:43)</a>:</h4>
<p>there are two services that connect to the production database from outside our VPC</p>



<a name="208811124"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208811124" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208811124">(Sep 02 2020 at 11:43)</a>:</h4>
<ul>
<li>the perf collector</li>
<li>rust-bots for the zulip archives generation</li>
</ul>



<a name="208816022"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208816022" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208816022">(Sep 02 2020 at 12:31)</a>:</h4>
<p>zulip archives shouldn't need database?</p>



<a name="208816034"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208816034" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208816034">(Sep 02 2020 at 12:31)</a>:</h4>
<p>perf collector is a problem though</p>



<a name="208816064"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208816064" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208816064">(Sep 02 2020 at 12:31)</a>:</h4>
<p>I'd be down for ssh opening a hole there</p>



<a name="208817416"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208817416" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208817416">(Sep 02 2020 at 12:44)</a>:</h4>
<p><span class="user-mention" data-user-id="116122">@simulacrum</span> hmm? why does the collector need database access?</p>



<a name="208817443"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208817443" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208817443">(Sep 02 2020 at 12:44)</a>:</h4>
<p>it records everything into the database?</p>



<a name="208817479"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208817479" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208817479">(Sep 02 2020 at 12:44)</a>:</h4>
<p>like, the alternative would be to provide APIs on the frontend to write into the database, but that's really annoying because we use transactions intentionally right now</p>



<a name="208817537"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208817537" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208817537">(Sep 02 2020 at 12:45)</a>:</h4>
<p>hmm yeah crater uses an api to avoid giving database access to the agents</p>



<a name="208817861"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208817861" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208817861">(Sep 02 2020 at 12:48)</a>:</h4>
<p>sure, but that causes the 100% CPU usage on the rust-bots crater server, at least in part, because an API is annoying to use transactions around</p>



<a name="208817884"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208817884" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208817884">(Sep 02 2020 at 12:48)</a>:</h4>
<p>and in any case perf isn't really okay with losing transactions</p>



<a name="208817919"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208817919" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208817919">(Sep 02 2020 at 12:49)</a>:</h4>
<p>I'd be fine with an ssh tunnel though? So it seems like we can still do this</p>



<a name="208825313"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208825313" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208825313">(Sep 02 2020 at 13:43)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116122">simulacrum</span> <a href="#narrow/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet/near/208816022">said</a>:</p>
<blockquote>
<p>zulip archives shouldn't need database?</p>
</blockquote>
<p>in the security group there is this comment:</p>
<blockquote>
<p>Connections from rust-bots ec2, used for archive generation</p>
</blockquote>



<a name="208825461"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208825461" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208825461">(Sep 02 2020 at 13:44)</a>:</h4>
<blockquote>
<p>I'd be fine with an ssh tunnel though? So it seems like we can still do this</p>
</blockquote>
<p>it kinda seems more fragile than keeping rds in a public subnet</p>



<a name="208826705"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208826705" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208826705">(Sep 02 2020 at 13:54)</a>:</h4>
<p><span class="user-mention" data-user-id="121055">@Pietro Albini</span> oh, that was referring to the rustc-pef sqlite database export that has been disabled</p>



<a name="208826753"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208826753" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208826753">(Sep 02 2020 at 13:54)</a>:</h4>
<p>(because it hogs CPU and network bandwidth)</p>



<a name="208826759"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208826759" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208826759">(Sep 02 2020 at 13:54)</a>:</h4>
<p>ooh ok</p>



<a name="208826797"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208826797" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208826797">(Sep 02 2020 at 13:54)</a>:</h4>
<p>I'll remove the security group rule</p>



<a name="208826910"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/terraform%20with%20rds%20in%20a%20private%20subnet/near/208826910" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/terraform.20with.20rds.20in.20a.20private.20subnet.html#208826910">(Sep 02 2020 at 13:55)</a>:</h4>
<p>yeah maybe just public subnet is the better way then</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>